1. Technical Field
The present invention relates in general to data processing systems and in particular to memory management within data processing systems. Still more particularly, the present invention relates to a system, method and computer program product for automatically distributing real memory between virtual memory page sizes when all real memory of a system is in use.
2. Description of the Related Art
With the increasing penetration of processor-based systems into every facet of human activity, demands have increased on the processor and application-specific integrated circuit (ASIC) development and production community to produce systems that are able to store and process ever-increasing amounts of data at ever-higher speeds. Circuit products, including microprocessors, digital signal and other special-purpose processors, and ASICs, have become involved in the performance of a vast array of critical functions, and the involvement of microprocessors in the important tasks of daily life has heightened the expectation of instantaneous results. Whether the impact of delay in calculations would be measured in human lives or in mere dollars and cents, consumers of circuit products have lost tolerance for results that do not appear instantaneously.
The increasing emphasis on speed of the result has resulted in a concurrent emphasis on computing device performance. While users tend to think of performance in terms of processor speed, installed physical memory and graphics cards, other factors, such as the effectiveness of resource allocation in the use of virtual memory, can also prove critical. Virtual memory refers to the manipulation of a data structure called a paging file. The paging file is very closely related to the physical RAM installed in the computer. Its purpose is to complement the physical RAM and make increased storage available to the system. Both services and installed applications can benefit from this ‘extra’ RAM, even though it is substantially different from the physical RAM that tends to dominate discussions of storage.
When the load imposed by applications and services running on a computer nears the amount of installed RAM, an operating system needs additional RAM. When no additional RAM is available, the data processing system employs a substitute, such as virtual memory. The page file is created during operating system installation and resides on a hard drive. Page files are measured in megabytes. The size of the page file is based on the amount of RAM that is installed in the computer. In general, a page frame refers to a fixed piece of real memory (e.g. RAM) that serves as a container for a virtual page. A virtual page is a fixed-size unit of data that can exist either in memory (inside of a page frame) or on disk in paging space. A virtual page can be paged out (moved out of a page frame into paging space on disk) or paged in (moved into a page frame from paging space on disk). By default, some operating systems create a page file which is 1.5 times larger than the amount of installed physical RAM.
To improve system performance, many computer architectures and operating systems implement paging files with multiple page sizes. One significant problem with the use of multiple page sizes lies in determining how to allocate memory to each page size. For example, on a system with two page sizes of 4k and 64k, a decision must be made as to the allocation between 4k and 64k pages. The robustness and flexibility of systems currently in use are undermined by the inability under the prior art to dynamically adjust the allocation of virtual memory between various page sizes when all real memory is in use.
In situations in which all of the real memory of a system is not in use (i.e., the virtual memory footprint of a system's workload is smaller than the real memory size of the data processing system), the problem of memory allocation is simplified, because an operating system can automatically adjust the amount of memory to use for each page size based on the amount of memory that is “free” in each page size. For example, in a system with 64k and 4k pages, if 50% of the 64k pages on the system are not in use and 3% of the of the 4k pages are not in use, an operating system can be programmed under the prior art to shift memory from 64k pages to 4k pages. A page frame is a block of physical storage or RAM allocated to support a page.
The significant and, under the prior art, inadequately answered challenge for automatically distributing real memory across page sizes arises when all real memory is in use (i.e., the virtual memory footprint of a system's workload is larger than the real memory size of the data processing system). In such a case, an operating system cannot use the amount of free memory of each page size as the basis of a heuristic for allocating memory, because all of the available memory is in use.
For example, in a system with 64k and 4k pages, there could be two jobs using 4k pages and two jobs using 64k pages. The two 4k jobs could have consumed all of the available 4k pages on the system by scanning through cached file data that is used only once. The two 64k jobs may have used all available 64k pages with data that is highly referenced by the 64k jobs, and due to the thrashing on the 64k pages, these jobs may run slowly. In such an environment, the total throughput on the system suffers, and the total throughput would increase if an operating system increased the number of 64k pages on the system to reduce thrashing of 64k pages by 64k jobs. Because the 4k jobs do not re-reference cached file data, they don't need to keep the cache file data in memory, and reducing the number of 4k pages would have no negative effect on system performance.
The challenge to an operating system, which is not solved by the prior art, is detecting a situation such as that described immediately above, and adjusting the amount of memory allocated to different page sizes to improve a system's overall performance. What is needed is a system, method and computer program product for automatically distributing real memory between virtual memory page sizes when all real memory of a system is in use.